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ABSTRACT 

We  simulated  a  military  tank  environment  and  examined  the  performance  of  the  combined  position  of  gunner 
and  robotics  operator  and  how  aided  target  recognition  (AiTR)  capabilities  with  imperfect  reliability  (i.e., 
miss-prone  or  false-alarm-prone )  for  the  gunnery  task  might  affect  the  concurrent  robotics  and 
communication  tasks.  Specifically,  we  investigated  whether  performance  was  affected  by  individual 
differences  in  attentional  control  and  spatial  ability.  Results  showed  that  when  the  robotics  task  was  simply 
monitoring  the  video,  participants  had  the  best  performance  in  their  gunnery  and  communication  tasks  and 
the  lowest  perceived  workload,  compared  with  the  other  robotics  tasking  conditions.  There  was  a  strong 
interaction  between  the  type  of  AiTR  unreliability  and  participants'  perceived  attentional  control.  Overall,  for 
participants  with  higher  perceived  attentional  control,  false-alarm-prone  alerts  were  more  detrimental;  for 
low  attentional  control  participants,  conversely,  miss-prone  automation  was  more  harmful.  Consistent  with 
previous  findings,  low  spatial  ability  participants  preferred  visual  cueing,  and  high  spatial  ability  participants 
favored  tactile  cueing. 


1.0  INTRODUCTION 

In  this  study,  we  simulated  a  military  tank  environment  and  examined  the  performance  of  the  combined 
position  of  gunner  and  robotics  operator  and  how  aided  target  recognition  (AiTR)  capabilities  with  imperfect 
reliability  for  the  gunnery  task  might  affect  the  concurrent  robotics  and  communication  tasks.  Specifically,  we 
investigated  whether  performance  was  affected  by  individual  differences  factors  such  as  perceived  attentional 
control  (PAC)  and  spatial  ability  (SpA).  We  followed  the  paradigm  of  Chen  and  Terrence  [1]  and  manipulated 
the  reliability  of  the  AiTR  cueing  system  (i.e.,  false-alarm  [FA]  and  miss  rates).  Based  on  the  data  from 
Wickens,  Dixon,  Goh,  and  Hammer  [2],  we  expected  that  the  operator’s  gunnery  (automated)  task 
performance  would  degrade  if  the  FA  rate  of  the  AiTR  for  the  gunnery  system  was  high  because  of  reduced 
compliance  with  the  automation.  Conversely,  if  the  cueing  was  miss-prone  (MP),  the  operator’s  robotics 
(concurrent)  task  performance  would  be  affected  more  than  the  gunnery  task  because  of  reduced  reliance  on 
the  automation.  More  mental  and  visual  resources  would  be  devoted  to  checking  the  raw  data  for  the 
automated  task,  and  therefore,  the  performance  of  the  concurrent  task  would  be  degraded.  On  the  other  hand, 
there  was  evidence  that  FAP  automation  was  more  detrimental  to  both  the  automated  and  concurrent  tasks 
than  MP  automation  [3].  Therefore,  it  is  likely  that  false-alarm-prone  (FAP)  automation  would  have  a  more 
negative  impact  on  the  overall  performance  than  would  MP  automation.  In  other  words,  there  have  been 
conflicting  results  in  the  literature  regarding  the  independence  of  the  effects  of  FAP  and  MP  automation  on 
operator  compliance  and  reliance.  It  is  possible  that  individual  differences  may  be  responsible  for  some  of  the 
observed  differences  in  the  literature.  Therefore,  we  investigated  the  effects  of  individual  differences  on  FAP 
and  MP  conditions  as  a  possible  explanation  for  the  discrepancies. 
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In  the  current  study,  we  sought  to  investigate  the  effects  of  individual  differences  in  SpA  and  PAC  on  the 
operators’  concurrent  performance.  Several  studies  show  that  there  are  individual  differences  in  multitasking 
performance,  and  some  people  are  less  prone  to  performance  degradation  during  multitasking  conditions 
[4] [5].  There  is  some  evidence  that  attention-switching  flexibility  can  predict  performance  of  such  diverse 
tasks  as  flight  training  and  bus  driving  [6].  There  is  also  evidence  that  people  with  better  attention  control  can 
allocate  their  attention  more  flexibly  and  effectively  [7]  [8],  and  this  was  partially  confirmed  by  Chen  and 
Joyner  [9].  It  is  likely  that  operators  with  different  levels  of  attention  switching  abilities  may  react  differently 
to  automated  systems  with  FAs  and  misses.  In  other  words,  operators’  compliance  and  reliance  behaviors  may 
be  altered  based  on  their  ability  to  effectively  switch  their  attention  among  the  systems.  For  example,  the 
complacency  effect  may  be  more  severe  for  poor  attentional  control  individuals  compared  with  those  with 
better  attentional  control.  The  current  study  sought  to  examine  if  the  compliance  vs.  reliance  effects  reported 
in  the  literature  might  be  moderated  by  individual  attentional  control. 

In  addition  to  PAC,  we  also  examined  the  relationship  between  operators’  SpA  and  their  performance  as  well 
as  the  ways  they  interact  with  the  AiTR  cueing  system.  Chen  and  Terrence  [1]  observed  that  low  SpA 
operators  indicated  a  preference  of  visual  cueing  display  over  tactile  display,  while  high  SpA  operators 
showed  the  opposite  preference.  The  current  study  provided  an  opportunity  to  examine  if  this  finding  could  be 
replicated.  Additionally,  Lathan  and  Tracey  [10]  demonstrated  that  people  with  higher  SpA  performed  better 
in  a  teleoperation  task  through  a  maze.  They  finished  their  tasks  faster  and  had  fewer  errors.  Lathan  and 
Tracey  suggested  that  military  missions  can  benefit  from  selecting  personnel  with  higher  SpA  to  operate 
robotic  devices.  Our  previous  studies  also  found  SpA  to  be  a  good  predictor  of  the  operator’s  robotics  and 
gunnery  task  performance  [1]  [9]  [1 1]. 


2.0  METHOD 

2.1  Participants 

Twenty-four  college  students  (4  females  and  20  males,  mean  age  =  22.3)  participated  in  this  study,  and  they 
were  compensated  $  15/hr  or  with  class  credit  for  their  participation. 

2.2  Apparatus 
2.2.1  Simulators 

The  experiment  was  conducted  using  Tactical  Control  Unit  (TCU)  developed  by  the  U.S.  Army  Research 
Laboratory’s  Robotics  Collaborative  Technology  Alliance  for  the  robotics  control  tasks.  The  TCU  is  a  one- 
person  crew  station  from  which  the  operator  can  control  several  simulated  robotic  assets,  which  can  either 
perform  their  tasks  semi-autonomously  or  be  teleoperated.  The  operator  switched  operation  modes  and  display 
modes  through  the  use  of  a  19  in.  touch-screen  display.  A  joystick  was  used  to  manipulate  the  direction  in 
which  the  robot  moved  when  in  Teleop  mode.  The  robot  simulated  in  our  study  is  the  experimental  unmanned 
vehicle  (XUV)  developed  by  the  Army  Research  Laboratory.  The  simulation  program  used  in  this  study  was 
rSAF,  which  is  a  version  of  OneSAF  for  robotics  simulation.  The  gunnery  component  was  implemented  using 
an  additional  screen  and  controls  to  simulate  the  out-the -window  view  and  line-of-sight  fire  capabilities.  The 
interface  consisted  of  a  15  in.  flat  panel  monitor  and  a  joystick.  Participants  used  the  joystick  to  rotate  the 
sensors  360°,  zoom  in  and  out,  and  engage  targets.  Figure  1  shows  the  user  interface  of  the  TCU  and  the 
gunnery  station. 
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Figure  1 :  Gunnery  station  (right)  and  TCU  (left). 

To  augment  target  detection  in  the  gunnery  component,  visual  and  tactile  alerts  were  used  to  cue  the 
participant  to  the  direction  of  a  target  as  determined  by  the  AiTR.  Visually,  the  targets  consisted  of  icons 
presented  around  the  overhead  view  diagram  of  the  participant  vehicle  in  the  lower  right  area  of  the  screen. 
The  target  icon  appeared  in  one  of  eight  possible  locations  around  the  gunner,  corresponding  to  45  degree 
increments  along  a  360°  azimuth.  As  the  gunner  rotated  the  view,  the  turret  portion  of  the  vehicle  diagram 
moved  along  the  eight  possible  orientations  to  allow  the  gunner  to  place  his/her  field  of  view  on  the  cued 
target.  Tactually,  target  positions  relative  to  the  gunner  were  presented  using  eight  electromechanical 
transducers  known  as  ‘tactors.’  The  eight  tactors  were  arranged  equidistantly  on  an  elasticized  belt  worn 
around  the  abdomen  just  above  the  navel. 

The  reliability  level  of  the  FAP  and  MP  alerts  was  both  60%.  The  low  reliability  level  was  deliberately  chosen 
to  investigate  if  the  compliance  vs.  reliance  effects  as  well  as  the  individual  differences  reported  previously  in 
the  literature  would  be  amplified  in  the  high  workload  multitasking  environment  in  the  current  study.  The 
FAP  condition  consisted  of  ten  hits  (i.e.,  alerts  when  there  were  targets),  eight  FAs  (i.e.,  alerts  when  there 
were  no  targets),  no  misses  (i.e.,  no  alerts  when  there  were  targets),  and  two  correct  rejections  (CRs)  (i.e.,  no 
alerts  when  there  were  no  targets).  The  MP  condition  consisted  of  two  hits,  no  FAs,  eight  misses,  and  ten  CRs. 
Additionally,  only  hostile  targets  were  cued,  not  the  neutral  targets.  The  participants  were  instructed  to  detect 
the  neutral  targets  independently.  The  alerts  (either  true  or  false)  did  not  occur  when  neutral  targets  appeared 
in  the  environment. 

2.2.2  Communication  Task  Materials 

The  communication  task  was  administered  concurrently  with  the  experimental  scenarios.  The  questions 
included  simple  military-related  reasoning  tests  and  simple  memory  tests.  The  inclusion  of  these  cognitive 
tasks  was  for  simulating  an  environment  where  the  gunner  was  communicating  with  fellow  crew  members  in 
the  vehicle.  Test  questions  were  delivered  by  a  synthetic  speech  program,  DECTalk®.  The  questions  were 
pre-recorded  by  a  male  speaker  and  presented  at  the  rate  of  one  question  every  33  seconds. 


RTO-MP-HFM-1 81 


P12-3 


Effects  of  Individual  Differences  on  Operators’  Interaction 

with  Imperfect  Automation  in  a  Simulated  Multitasking  Environment 


mttiAiUtat  Artt/rt  kt 


2.2.3  Questionnaires  and  Spatial  Tests 

A  questionnaire  about  attentional  control  [8]  was  used  to  evaluate  participants’  PAC.  The  attentional  control 
survey  consists  of  21  items  and  measures  perceived  attention  focus  and  shifting.  The  scale  has  been  shown  to 
have  good  internal  reliability  (a  =  .88).  In  our  previous  studies  [9],  we  observed  a  positive,  although 
somewhat  weak,  relationship  between  attentional  control  survey  score  and  some  multitasking  performance 
measures.  The  Cube  Comparison  and  the  Hidden  Patterns  tests  [12]  as  well  as  the  Spatial  Orientation  Test 
[13]  were  used  to  assess  participants’  SpA.  Participants’  workload  was  evaluated  using  the  computer-based 
version  of  NASA-TLX  questionnaire  [14],  A  usability  questionnaire  was  used  to  assess  participants’  reliance 
on  tactile  and/or  visual  cueing  for  the  gunnery  task.  Participants  rated  their  preference  on  a  5 -point  scale  (from 
1  to  5:  entirely  visual-  predominately  visual-  both  visual  &  tactile-  predominately  tactile-  entirely  tactile). 
Participants  were  also  asked  to  evaluate  their  trust  in  the  AiTR  system  using  a  modified  survey  by  Jian 
Bisantz,  and  Drury  [15]. 

2.2.4  Experimental  Design 

The  overall  design  of  the  study  is  a  2  x  3  mixed  design.  The  between -subject  variable  is  AiTR  type  (FAP  vs. 
MP).  The  within-subject  variable  is  Robotics  Task  type  (Monitor  vs.  Auto  vs.  Teleop)  (see  Procedure). 

2.3  Procedure 

After  the  informed  consent  process,  participants  were  administered  the  attentional  control  survey  and  the 
spatial  tests.  After  these  tests,  participants  received  training,  which  was  self-paced  and  was  delivered  by 
PowerPoint®  slides  showing  the  elements  of  the  TCU,  steps  for  completing  various  tasks,  and  exercises  for 
performing  the  robotic  control  tasks.  After  the  tutorial  on  TCU,  participants  were  trained  on  the  gunnery  tasks. 
The  entire  training  session  lasted  about  2.5  hr. 

In  the  experimental  trials,  participants’  tasks  were  to  use  their  robot  to  locate  hostile  targets  in  the  remote 
environment  and  also  find  targets  in  their  immediate  environment.  The  tank  was  simulated  as  traveling  along  a 
designated  route,  which  was  approximately  4.3  km  and  lasted  about  15  min.  There  were  10  hostile  and  10 
neutral  targets  along  the  route  in  each  gunnery  scenario.  Hostile  targets  were  enemy  soldiers  dressed  in 
military  uniform  and  carrying  a  gun;  neutral  targets  were  civilians  dressed  in  typical  Middle  Eastern  attire 
without  any  weapons.  Participants  were  instructed  to  engage  the  hostile  targets  and  verbally  report  spotting 
the  neutral  targets.  In  total,  there  were  three  15-min  scenarios,  corresponding  to  the  three  robotics  tasking 
conditions,  the  order  of  which  was  counterbalanced  across  participants. 

There  were  three  types  of  robotics  tasks:  Monitor,  Auto,  and  Teleop.  The  Monitor  task  required  the  operator 
to  continuously  monitor  the  video  feed  as  the  robot  traveled  autonomously  and  verbally  report  detection  of 
targets.  There  were  20  targets  (5  hostile  and  15  neutral)  along  the  route.  The  Auto  control  task  required  the 
operator  to  monitor  the  video  feed  as  the  robot  traveled  autonomously,  examine  still  images  generated  from 
the  reconnaissance  scans,  and  detect  targets.  The  Teleop  task  required  the  operator  to  manually  drive  the  robot 
(using  a  joystick)  along  a  predetermined  route  and  detect  targets.  For  both  the  Auto  and  Teleop  tasks,  there 
were  2  to  4  targets  (1  hostile  plus  1  to  3  neutral  targets)  at  each  checkpoint.  There  were  5  checkpoints  along 
the  designated  route.  Upon  detecting  a  target,  participants  needed  to  place  the  target  on  the  map,  label  the 
target,  and  then  send  a  spot  report. 

While  the  participants  were  performing  their  gunnery  and  robotics  control  tasks,  they  simultaneously 
performed  the  communication  task  by  answering  questions  delivered  to  them  via  DECtalk®.  There  were 
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2-min  breaks  between  experimental  scenarios.  Participants  assessed  their  workload  using  the  NASA-TLX 
after  each  scenario.  They  also  evaluated  their  perceived  utility  of  and  trust  in  the  AiTR  at  the  end  of  the 
experiment.  The  entire  experimental  session  lasted  about  1  hr. 

The  dependent  measures  include  mission  performance  (i.e.  number  of  targets  detected  in  the  remote 
environment  using  the  robot  and  number  of  hostile/neutral  targets  detected  in  the  immediate  environment), 
communication  task  performance,  and  perceived  workload. 


3.0  RESULTS 

3.1  Gunnery  Task  Performance 

A  mixed  analysis  of  variance  (ANOVA)  was  performed  to  examine  the  effects  of  the  concurrent  robotic 
control  tasks  on  the  gunnery  task  performance  (percentage  of  hostile  targets  detected),  with  the  AiTR 
condition  (FAP  vs.  MP)  being  the  between-subject  factor  and  the  Robotics  Task  condition  (Monitor  vs.  Auto 
vs.  Teleop)  as  the  within-subject  factor.  The  analysis  revealed  that  Robotics  condition  significantly  affected 
number  of  targets  detected,  F( 2,  15)  =  4.6,  p  <  .05  (Fig  2).  Post  hoc  (least  significant  difference  [LSD])  tests 
showed  that  target  detection  in  the  Monitor  condition  was  significantly  higher  than  in  the  Auto  and  Teleop 
conditions. 


Figure  2:  Gunner’s  target  detection  performance  (hostile  targets). 

Participants  were  classified  as  high  or  low  SpA  based  on  their  composite  scores  of  the  spatial  tests  (median 
split).  Participants  with  higher  SpA  had  significantly  higher  gunnery  task  performance  than  did  those  with 
lower  SpA,  F(l,  16)  =  6.3,  p  <  .05.  Participants  were  classified  as  high  or  low  PAC  based  on  their  attentional 
control  survey  scores  (median  split).  There  was  also  a  significant  AiTR  x  PAC  interaction,  F(\ ,  16)  =  7.4,  p  < 
.05  (Fig  3,  upper  left).  Those  with  lower  PAC  performed  better  with  the  FAP  cueing,  whereas  those  with 
higher  PAC  performed  at  a  similar  level  regardless  of  the  AiTR  conditions. 

In  order  to  further  examine  the  effect  of  task  load  on  reliance  of  AiTR,  the  data  of  the  MP  condition  were 
analyzed  separately.  Due  to  the  small  sample  size  (N  =  12),  no  significant  differences  were  found  between 
those  with  high  vs.  low  PAC,  F(l,  10)  =  1.4,  p  >  .05.  However,  the  trend  was  evident  that,  while  those  with 
high  PAC  maintained  a  fairly  stable  level  of  reliance  throughout  the  experimental  conditions,  those  with  low 
PAC  became  increasingly  reliant  on  the  AiTR,  as  task  load  became  heavier  (i.e.,  Teleop  >  Auto  >  Monitor, 
based  on  Chen  and  Joyner  [9])  (Fig  4).  For  the  low  PAC  participants,  the  difference  between  the  Monitor  and 
Teleop  conditions  was  statistically  significant,  F(l,  6)  =  7.1,  p  <  .05. 
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Figure  3:  Interaction  between  PAC  and  AiTR  unreliability. 


Figure  4:  Effects  of  PAC  on  gunnery  task  performance  (Miss-prone  condition). 


Participants’  detection  of  neutral  targets  was  also  assessed.  Since  the  AiTR  only  alerted  the  participants  when 
hostile  targets  were  present,  the  neutral  target  detection  could  be  used  to  indicate  how  much  visual  attention 
was  devoted  to  the  gunnery  station.  A  mixed  ANOVA  revealed  a  significant  main  effect  for  Robotics,  F(2, 1 5) 
=  4.4,  p  <  .05.  Post  hoc  tests  (LSD)  showed  that  neutral  target  detection  in  the  Teleop  condition  was 
significantly  lower  than  in  the  Auto  condition.  The  main  effect  for  AiTR  failed  to  reach  statistical 
significance,  F(l,  22)  =  3.3,  p  >  .05.  There  was  a  significant  AiTR  x  PAC  interaction,  F(  1 ,  1 6)  =  3.6,  p  <  .05 
(Fig  3,  upper  right).  Those  with  lower  PAC  performed  at  about  the  same  level,  regardless  of  the  AiTR  type, 
while  those  with  higher  PAC  had  a  better  performance  with  the  MP  cueing  than  with  the  FAP  cueing. 
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3.2  Robotics  Task  Performance 

A  mixed  ANOVA  revealed  that  there  was  a  significant  main  effect  for  Robotics,  F(2, 1 5)  =  25.4,  p  <  .001.  The 
Monitor  condition  was  significantly  higher  than  both  the  Auto  and  the  Teleop  conditions,  in  terms  of 
percentage  of  targets  detected.  The  main  effect  for  AiTR  was  not  significant,  p  >  .05.  There  was  a  significant 
Robotics  x  AiTR  interaction,  F( 2,32)  =  4.0,  p  <  .05.  The  Monitor  task  performance  stayed  at  the  same  level 
regardless  of  the  AiTR  types.  The  Auto  task  performance  was  slightly  higher  with  the  MP  cueing  (although 
the  difference  failed  to  reach  statistical  significance),  while  the  Teleop  task  performance  was  significantly 
higher  with  the  FAP  cueing  (p  <  .05).  There  was  also  a  significant  AiTR  x  PAC  interaction,  F(l,l 6)  =  4.8,  p  < 
.05  (Fig  3,  lower  left).  Those  with  lower  PAC  had  a  better  performance  with  the  FAP  cueing,  while  those  with 
higher  PAC  performed  better  with  the  MP  cueing. 

3.3  Communication  Task  Performance 

A  mixed  ANOVA  revealed  that  there  was  a  significant  main  effect  for  Robotics,  F(2,44)  =  3.2,  p  <  .05. 
Monitor  was  significantly  higher  than  Teleop,  F(  1 ,22)  =  5.5,  p  <  .05.  Neither  the  main  effect  for  AiTR  nor  the 
Robotics  x  AiTR  interaction  was  significant  (Fig  3,  lower  right). 

3.4  Operator  Perceived  Workload 

Participants’  self-assessment  of  workload  (weighted  ratings  of  the  scales  of  the  NASA-TLX)  was  significantly 
affected  by  Robotic  condition,  F(2, 1 5)  =  25.1,  p  <  .001.  The  perceived  workload  was  significantly  higher  in 
the  Teleop  condition  (M  =  77.7)  than  in  Auto  (M  =  69.6)  and  Monitor  (M  =  61.1).  The  difference  between 
Auto  and  Monitor  was  also  significant.  The  main  effect  for  AiTR  was  not  significant,  p  >  .05.  There  was  a 
significant  Robotics  x  AiTR  interaction,  F(2, 1 5)  =  5.5,  p  <  .05. 

3.5  AiTR  Display  Usability  Assessment 

Following  their  interaction  with  the  AiTR  systems,  41%  of  participants  responded  that  they  relied 
predominantly  or  entirely  on  the  tactile  AiTR  display,  while  36%  responded  that  they  relied  predominantly  or 
entirely  on  the  visual  AiTR  display.  AiTR  preference  was  also  significantly  correlated  with  SpA  (composite 
spatial  test  scores)  (r  =  .51,  p  <  .01).  Those  with  higher  SpA  tended  to  prefer  tactile  cueing  over  visual  cueing. 
Conversely,  those  with  lower  SpA  favored  visual  cueing  over  tactile  cueing.  We  also  evaluated  participants’ 
self-assessed  trust  in  the  AiTR  system  and  compared  those  with  higher  and  lower  PAC.  There  was  no 
significant  difference  between  these  two  groups,  p  >  .05. 


4.0  DISCUSSION 

In  this  study,  we  simulated  a  military  tank  environment  and  examined  the  performance  and  workload  of  the 
combined  position  of  gunner  and  robotics  operator.  More  specifically,  we  investigated  the  effects  of  AiTR 
with  imperfect  reliability  (FAP  vs.  MP)  on  operator’s  performance  of  the  automated  (i.e.,  gunnery)  task  as 
well  as  the  concurrent  tasks  (i.e.,  robotics  and  communication).  Our  results  showed  that  the  operator’s 
gunnery  task  performance  in  detecting  hostile  targets  was  significantly  better  in  the  Monitor  condition  than  in 
the  other  two  robotics  task  conditions,  consistent  with  the  findings  of  Chen  and  Joyner  [9].  In  both  Chen  and 
Joyner  [9]  and  the  current  study,  the  workload  associated  with  the  Monitor  condition  was  significantly  lower 
than  the  other  robotics  conditions.  These  results  suggest  that  the  operator  had  more  visual  and  mental 
resources  for  the  gunnery  task  when  the  robotics  task  was  simply  monitoring  the  video  feed,  compared  with 
the  other  two  robotics  conditions. 
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Also  consistent  with  the  previous  findings  [1][9],  participants’  SpA  was  found  to  be  an  accurate  predictor  of 
their  gunnery  performance.  Thomas  and  Wickens  [16]  showed  that  there  were  individual  differences  in 
scanning  effectiveness  and  its  associated  target  detection  performance.  However,  Thomas  and  Wickens  did 
not  examine  the  characteristics  of  those  participants  who  had  more  effective  scanning  strategies.  The  findings 
of  the  current  study  along  with  the  previous  two  studies  indicate  that  SpA  may  be  an  important  factor  for 
determining  scanning  effectiveness. 

Our  results  also  showed  that  there  was  a  significant  interaction  between  types  of  unreliable  AiTR  and 
participants’  PAC.  Taking  the  three  main  performance  measures  together  (i.e.,  Gunnery-  Hostile,  Gunnery- 
Neutral,  and  Robotics),  it  appears  that  overall,  for  high  PAC  participants,  FAP  alerts  were  more  detrimental 
than  MP  alerts.  FAP  alerts  not  only  affected  their  automated  task  but  also  the  concurrent  task.  This  finding  is 
consistent  with  the  conclusion  of  Dixon,  Wickens,  and  McCarley  [3]  that  FAP  degraded  overall  performance 
more  than  MP  automation.  However,  it  is  worth  noting  that  for  low  PAC  participants,  we  observed  the 
opposite  pattern:  MP  automation  was  more  harmful  than  FAP  automation.  Based  on  Figure  3,  it  is  evident  that 
high  PAC  participants  did  not  comply  with  alerts  in  the  FAP  condition.  Since  the  FAP  AiTR  had  a  0%  miss 
rate,  a  full  compliance  should  result  in  a  detection  rate  over  84%,  as  reported  in  Chen  and  Terrence  [1]  (in 
which  a  perfectly  reliable  AiTR  was  used).  As  predicted,  Figure  3  shows  that  in  MP  conditions,  high  PAC 
participants  did  not  rely  on  the  AiTR  and  detected  more  targets  than  were  cued.  However,  an  examination  of 
the  data  for  the  low  PAC  participants  revealed  a  completely  opposite  trend.  Specifically,  with  the  FAP 
condition,  low  PAC  participants  showed  a  strong  compliance  with  the  alerts,  which  resulted  in  a  good 
performance  in  target  detection  (at  a  similar  level  as  in  Chen  &  Terrence  [1]).  With  the  MP  condition, 
however,  low  PAC  participants  evidently  overly  relied  on  the  automation  and  therefore  had  a  very  poor 
performance.  Indeed,  Figure  4  shows  that  as  task  load  became  heavier,  those  with  low  PAC  became 
increasingly  reliant  on  the  AiTR,  while  those  with  high  PAC  maintained  a  fairly  stable  level  of  reliance 
throughout  the  experimental  conditions.  According  to  Biros,  Daly,  and  Gunsch,  [17],  higher  task  loads  tend  to 
induce  a  higher  level  of  reliance  on  automated  systems.  Our  present  data  suggest  that  this  heightened  level  of 
reliance  is  also  moderated  by  PAC.  More  specifically,  only  those  with  low  PAC  tend  to  exhibit  over-reliance 
on  automation  (i.e.,  complacency)  under  a  heavy  task  load. 

The  overall  data  suggest  that  low  PAC  participants  had  a  higher  trust  in  the  automation  system  than  did  high 
PAC  participants.  It  is  likely  that  low  PAC  participants  had  more  difficulty  in  performing  multiple  tasks 
concurrently  and  had  to  rely  on  automation  when  available.  High  PAC  participants,  in  contrast,  tended  to  rely 
on  their  own  multitasking  ability  to  perform  the  tasks.  It  is  interesting  to  note  that  there  was  no  significant 
difference  in  the  participants’  self-assessment  of  their  trust  in  the  AiTR  system  between  high  PAC  and  low 
PAC  groups.  This  suggests  that  the  participants’  self-assessed  trust  in  automation  may  not  truly  reflect  their 
actual  use  (i.e.  actual  trust)  of  automation.  Our  results  are  consistent  with  past  research  [  1 8] [  19]  that  self- 
confidence  is  a  critical  factor  in  moderating  the  effect  of  trust  (in  automation)  on  reliance  (on  the  automatic 
system).  Lee  and  Moray  [19]  found  that  when  self-confidence  exceeded  trust,  operators  tended  to  use  manual 
control.  When  trust  exceeded  self-confidence,  automation  was  used  more.  Our  present  data  suggest  that,  this 
relationship  between  self-confidence  and  level  of  reliance  is  also  moderated  by  operator’s  PAC. 

Participants’  communication  task  performance  was  significantly  better  when  their  robotics  task  was  Monitor 
than  when  it  was  Teleop.  This  finding  is  consistent  with  Chen  and  Joyner  [1],  Participants’  perceived 
workload  was  found  to  be  affected  by  the  type  of  concurrent  robotics  task.  The  workload  was  significantly 
higher  in  the  Teleop  condition  than  in  the  Auto  condition,  which  in  turn  was  significantly  higher  than  in  the 
Monitor  condition.  These  results  are  consistent  with  Mitchell’s  [20]  analysis  and  with  the  findings  of  Chen 
and  Joyner  [9],  Chen  and  Terrence  [1],  and  Schipani  [21],  which  evaluated  robotics  operator  workload  in  a 
field  setting. 
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The  significant  positive  correlation  of  AiTR  preference  with  SpA  indicated  that  as  AiTR  ratings  tended 
toward  considerable  reliance  on  the  tactile  display,  there  was  a  concurrent  shift  with  higher  SpA.  This 
confirmed  a  trend  that  was  first  observed  in  Chen  and  Terrence  [1].  Perhaps  those  with  higher  SpA  can  more 
easily  employ  the  spatial  tactile  signals  in  the  dual  task  setting  and  therefore  have  a  stronger  preference  for 
something  that  makes  the  gunner  task  easier  to  complete.  Individuals  with  lower  SpA,  on  the  other  hand,  may 
have  not  utilized  the  spatial  tactile  cues  to  their  full  extent  and  therefore  continued  to  prefer  the  visual  AiTR 
display.  According  to  Kozhevnikov,  Hegarty,  and  Mayer  [22],  visualizers  with  lower  SpA  tend  to  rely  on 
iconic  imagery  while  those  with  higher  SpA  tend  to  prefer  using  spatial-schematic  imagery  while  solving 
problems.  Therefore,  it  is  likely  that  in  our  study,  those  who  preferred  visual  AiTR  displays  might  be  more 
iconic  in  their  mental  representations.  However,  this  preference  may  have  caused  degraded  target  detection 
performance  due  to  more  visual  attention  being  devoted  to  the  visual  AiTR  display,  not  to  the  simulated 
environment.  In  contrast,  those  who  were  more  spatial  relied  on  the  directional  information  of  the  tactile 
display  to  help  them  with  the  visually  demanding  tasks,  resulting  in  a  more  effective  performance. 
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